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Abstract 

In this paper we discuss the notion of singular value tuples of a complex 
valued rf-mode tensor of dimension mi x . . . x m^. We show that a general tensor 
has a finite number of singular value tuples, viewed as points in a corresponding 
Segre variety. We give the formula for the number of singular value tuples. We 
show similar results for tensors with partial symmetry. We give analogous 
results for the homogeneous pencil eigenvalue problem for cubic tensors, i.e. 
mi = ... = rrid. We show that best rank one approximations for general 
tensors are unique. Similarly, for general partially symmetric tensors the best 
rank one approximation is unique and partially symmetric. We show that a 
best rank-(ri, . . . , r^) approximation for a general d-mode tensor is unique. 
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1 Introduction 

The object of this paper is to study two closely related topics: counting the number 
of singular vector tuples of a d-mode complex generic tensor of dimension mi x 
. . . X 171(1 and the uniqueness of its best rank one approximation. To motivate our 
results let us consider the classical case of matrices, i.e. d = 2 and A G M"*i^"*2 = 
]^mi ^ ]^m2_ We call a pair (xi,X2) G (M"*! \ {0}) x (M"^^ \ |o}) a singular vector 
pair if 

^X2 = AlXi, ^"""xi = A2X2, (1.1) 

for some Ai, A2 G M. For x G M™ let ||x|| := Vx^ be the Euclidean norm on M™. 
Choosing xi,X2 to be of Euclidean length one we deduce that Ai = A2, where |Ai| 
is equal to some singular value of A. It is natural to identify all singular vector 
pairs of the form (04X1,02X2), where 0102 7^ as the class of singular vector pair. 
This is equivalent to view a singular vector pair as an element of P(]R™i) x P(M™'2)^ 
where P(F™') is the space of all lines through the origin in F™ for any field F. For 
a generic A, i.e. A of the maximal rank r = min(mi,m2) and r distinct positive 
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singular values, A has exactly r distinct singular vector pairs. Furthermore, under 
these conditions A has a unique best rank one approximation in the Frobenius norm 
given by the singular pair corresponding to the maximal singular value [8j. 

Assume now that m = mi = m2 and j4 is a real symmetric matrix. Then the 
singular values of A are the absolute values of the eigenvalues of A. Furthermore, 
if all the absolute values of the eigenvalues of A are pairwise distinct then A has 
a unique best rank one approximation which is symmetric. Hence for any real 
symmetric matrix A there exists a best rank one approximation which is symmetric. 

In this paper we derive similar results for tensors. To state more precisely 
our results we introduce some notation. Let F be either the real field M or the 
complex field C. Let mj > 2 be an integer for i G [d] := {l,...,d}. Denote 
m := (mi, . . . , 171^). Let i;F(m) := P(F'"i) x . . . x P(F™<*) be the Segre variety over 
F. Set S(m) := Sc(m). 

Denote by F™ = ]p™ix---x™d — igj^L-^F"^* the vector space of d-mode tensors 
T = = = over F. (We assume that d > 3 unless 

stated otherwise.) For an integer p G [d] and for Xj^ G F™J'-,r G [p] we use the 
notation '^j^,r&[p]^jr '■— ^ji <K> . . . (E> Xj^. For a subset P = {ji, ■ ■ ■ , jp} ^ [d] of 
cardinality p = \P\, consider a p-mode tensor X = [a^j^^,...,^^^] G 'i^jr,r€[p]^^^'' i where 
ji< ...< jp. Define T x X := J2i^^(^[m,^],r&[p] th,...,iaXij^,...,i,^ to be a (d - p)-mode 
tensor obtained by contraction on the indices ij^ , . . . , ij^ . 

Let T G F"^. We first define the notion of a singular vector tuple (xi, . . . ,X(i) G 
(F"^! \ {0}) X ... X (F™'' \ {0}) [H]: 

T X ^je[d]\{t}^j = AjXj, i = l,...,d. (1.2) 

As for matrices we identify all singular vector tuples of the form (aixi, . . . ,a(iXrf), 
ai . . . a^; 7^ as one class of singular vector tuple in S]f (m). (Note that for d = 2 
and F = C our notion of singular pair differs from the classical notion of singular 
vectors for complex valued matrices.) We remark that, as in the classical SVD, F™'* 
cannot be replaced by a vector space Vi of dimension mj, unless we choose a basis 
which allows to identify Vi with its dual . Our first major result is that for a 
generic tensor T G the number of singular vectors tuples in (xi, . . . , x^) G S(m) 
is finite and it is equal to c(m). c(m) is the coefficient of the monomial Iljefd] 
in the homogenous polynomial in variables ti, . . . ,t(i 

n V I; ' t„te[d]. (1.3) 

ie[d] ' ' je[d]\{t} 

In particular, for a generic T G ]R™ix --xmtj number of singular vectors tuples in 
(xi, . . . , Xd) G P(M™i) X ... X P(M™'^) is finite and it is at most c(m). At the end of 
^we list the ffist values of c(m) for d = 3. We generalize the above results for the 
class of tensors with given partial symmetry. 

We now consider the cubic case where mi = . . . = m^ = m. For an integer m > 2 
let m^'^ := (m, . . . ,m). Then T G F™ is called d-cube, or simply a cube tensor. 



For a vector x G C™ let x := x (g) . . . (g) x. Assume that T, 5 G C" . Then the 

V ' 

k 

homogeneous pencil eigenvalue problem is Tx(g)''^^x = XSx^'^^^jc. The contraction 
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here is with respect to the last d—1 indices of T, S respectively. We assume without 
the loss of generality that T = [tii,...,ij],S = are symmetric with respect 

to the indices i2,...,id- S is called nonsingular if the system S x = 

implies that x = 0. Assume that S is nonsingular and fixed. Then T has exactly 
m{d — 1)™"^ eigenvalues counted with their multiplicities. T has m{d — l)*""^ 
distinct eigenvectors eigenvectors in P(C™) for a general 7~. See [T^ for the case S 
is the identity tensor. 

View ]K'"ix---"^<i as an inner product space, where for two d-mode tensors T,S £ 
]gmix...xmd^^gjg^ ^^^^jj 'J- ^ ^ r^Y^^^ ^.j^g Hilbert-Schmidt norm is defined 
l|T|| := \/ {T, T) ■ (For simplicity of notation, we call the Frobenius and the Hilbert- 
Schmidt norm L2 norms.) The best rank one approximation is a solution to the 
minimal problem 

min ^ JIT- ®ie[d]Xi|| = ||r- (»ie[d]Ui||. (1.4) 

<^i(=[d]Ui is called best rank one approximation of T. In this paper we show that for 
a generic T G ]K'^ix --x'"d the best rank one approximation is unique. 

We generalize the above results for the class of tensors with given partial sym- 
metry. In particular we show that a generic real symmetric tensor T has a unique 



rank one approximation which satisfy (1.4). Hence this unique rank one tensor is 



symmetric. This gives another proof that the best rank one approximation to a 
symmetric tensor can be chosen symmetric [5]. (See a related result [21j.) 

In the last section of this paper we study a best rank-(ri, . . . , r^) approximation 
for a d-mode tensor [1]. We show that for a general tensor a best rank-(ri, . . . ^r^) 
approximation is unique. 

We now describe briefly the contents of our paper. In §2 we give layman's in- 
troduction to some basic notions of vector bundles over compact complex manifolds 
and Chern classes of certain bundles over the Segre varieties needed for this paper. 
We hope that this introduction will make our paper accessible to a wider audience. 
§3 discusses the first main contribution of this paper. Namely, the number of singu- 
lar value tuples of a general complex tensor is finite and is equal to c(mi, . . . , m^). 
We give a closed formula for c(mi, . . . , m^). §4 generalizes these results to partially 
symmetric tensors. In particular we reproduce the result of Cartwright and Sturm- 
fels for symmetric tensors 12]. In §5 we discuss a homogeneous pencil eigenvalue 
problem. In §6 we give certain conditions on a very general best approximation 
problem in M", which are probably well known to the experts. §7 give uniqueness 
results on best rank one approximation of general partially symmetric tensors. In 
§8 we discuss a best rank-(ri, . . . , r^) approximation. 



2 Vector bundles over compact complex manifolds 

In this section we recall some basic results on complex manifolds and holomorphic 
tangents bundles that we use in this paper. Our object was to give the simplest 
possible intuitive description of basic results in algebraic geometry needed in this 
paper, sometimes compromising the rigor. An interested reader can consult for 
more details with [9] for general facts about complex manifolds and complex vector 
bundles, and for a simple axiomatic exposition on complex vector bundles with |13|. 
For Bertini's type theorem we refer to Fulton [6j and Hartshorne |lDj. 
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2.1 Complex compact manifolds 

Let M be a compact complex manifold of dimension n. Thus there exists a finite 
open cover {Ui},i G [N] with coordinate homeomorphism <j)i : Ui ^ C" such that 
4>i ° is holomorphic (/>j(fj fl Uj) for all i^j. 

As an example consider the m — 1 dimensional complex projective space P(C™), 
which is the set of all complex lines in C"* through the origin. Any point in 
P(C"^) is represented by a one dimensional subspace spanned by the vector x = 
(xi, . . . , Xjn)^ G C™" \ {0}. The standard open cover of P(C™) consists of m open 
covers Ui, . . . , Um, where Ui corresponds to the lines spanned by x with Xi 7^ 0. The 
homeomorphism (f)i is given by (f)i{x.) = . . . , ^)^. So each Ui is 

homeomorphic to C"*"^. 

Let M be an n-dimensional compact complex manifold as above. For ( ^ Ui, the 
coordinates of the vector ^j(^) = z = (zi, . . . , z^,)^ are called the local coordinates 
of C- Since C" = M^", M is a real manifold of real dimension 2n. Let Zj = 
Xj + iyj,Zj = Xj — iyj,j G [n], where i = \/— T. For simplicity of notation wc let 
u = (ui, . . . ,U2n) = {xi,yi, . . . , Xn,yn) be the real local coordinates on Ui. Any 
function / : f/j C in the local coordinates is viewed as /(u) = ^(u) + i/i(u), where 
h, g : Ui ^ M.. Thus df = J2j&[2n] ^'^^ ^ positive integer p, a (differential) 

p-form oj on Ui is given in the local coordinates as follows 

w= XI /ii,..,ij,(u)duii A . . . Aduij,. 
l<ii<...<ip<2n 

Recall that the wedge product of two differential is anti commutative, i.e. dukAdui = 
—dui A duk- Then 

duj= X {dfi^^,„^i^)Adui^A...Aduip. 
l<ii<...<ip<2n 

(0-differential form is a function.) Note that for p > 2n any p differential form is a 
zero form. A straightforward calculation shows that d{du!) = 0. a; is a p form on M 
if its restriction to each U is an p form, and the restrictions of these two forms on 
Ui n Uj are obtained one from the other one by the change of coordinates 0^ o 0T^. 
cj is called closed if doj = 0, and doj is called an exact form. The space of closed 
p forms modulo exact p forms is a finite dimensional vector space over C, which 
is denoted by W{M). Each element of H^(M) is represented by a p closed form, 
and the difference between two representatives is an exact form. Since the product 
of two forms is also a form, it follows that all closed forms modulo exact forms is 
a finite dimensional algebra, where the identity 1 corresponds to constant function 
with value 1 on M. 

2.2 Holomorphic vector bundles 

A holomorphic vector bundle E on M oi rank k, where k is a nonnegative integer, 
is a complex manifold of dimension n + k which can be simply described as follows. 
There exists a finite open cover {Ui},i G [N] of M with the properties as above 

satisfying the following additional conditions. At each C U wc are given k- 
dimensional vector space E(^, called a fiber of E over which all can be identified 
with a fixed vector space Vj, having a basis [ei^i, . . . , efc^j]. For ( & UiCiUjji j the 
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transition matrix from [ei^j, . . . , e^^j] to [eij, . . . , e^j] is given by an k x k invertible 
matrix gu.uM)- So [ei,i, . . .,ek,i\ = [eij, . . . ,ekj]gUjU,{0- Each entry of gu^uAO 
is a holomorphic function in the local coordinates of Uj. We have the following 
relations 

auiUjiOgujuM) = 9u,Uj{v)9u,Up{v)9UpU^{v) = h for C G n Uj,ri e f/i n Uj n Up. 

{Ik is an identity matrix of order k.) 

For k = Q E is called a zero bundle. E is called a line bundle \i k = 1. E is 
called a trivial bundle if there exists a finite open cover such that each gUiUjiC) is 
an identity matrix. A vector bundle F on M is called a subbundle of if F is a 
submanifold of E such that is a subspace of E(^ for each ( G M. Assume that 
F is a subbundle of E. Then G := E/F is the quotient bundle of E and F, where 

is the quotient vector space Ec^/F,^. Let Ei,E2 be two vectors bundles on M. 
We can create the following new bundles on M. E := Ei® E2, F := Ei<^ E2 where 
Ei^ = Ei^^ © E2^(,F^ = Ei^f^ (8) £'2,^- particular, for a given vector bundle E on M 
we can define the bundle F := ®'^E. Here Fq = ®'^E(^ is a fiber of d-mode tensors. 

We now discuss a basic example used in this paper. Consider the trivial bundle 
E on P(C™') of rank m. So E(^ = C"^, and by abuse of notation we denote E as 
C™. The tautological line bundle T{m) on P(C''"), customary denoted by 0(— 1), 
is given by T(m)[x] = span(x) C C", where x is a nonzero vector spanning the line 
through zero that represents a point [x] in P(C™). So T(m) is a subbundle of C™. 
Denote by Q{m) the quotient bundle C"^ /T{m). So rank Q{m) = m — 1. We have 
an exact sequence of the following bundles on P(C™) 

^ T(m) ^ C™ ^ Q{m) 0. (2.1) 

The dual of the subbundle T{ra) of C™, also called the hyperplane line bundle, 
denoted here by H{m), associates with a point [x] S P(C'") the hyperplane y E 
(j-^mV^^Ty _ Q (Iqi^^ remind that the initial choice of the standard basis identifies 
C™" with its dual). H{m) is customary denoted by 0(1). 



2.3 Chern polynomials 

We now return to a holomorphic vector bundle F on a compact complex manifold 
M. One associate with E the Chern polynomial C{t,E) = 1 + ^ Cj{E)t^ (see 

pug). Here Cj{E) e R'^^{M). Note that Cj{E) = for j > 2n. C{t,E) = 1 ii E 
is a trivial line bundle. C{t,Ei © £"2) = C{t, Ei)C{t, E2), (Whitney sum formula). 
Furthermore, for a pull back bundle tt^Ei described above we have the equality 
Cj^nlEi) = Cj{Ei), when we use the local coordinates C = (Ci; C2) on Mi x M2. 

We now summarize the information on Chern classes needed in this paper. 
First, on P(C™) Chern classes may be identified with integers, and we can write 
C{t,T{m)) = 1 — t,C{t, H{m)) = 1 + t. Moreover the algebra of all closed forms 
modulo the exact forms on P(C™') is C[t]/(t™), i.e. all polynomials in the variable 



t modulo the relation i*" = 0. The exact sequence (2.1) yields that C{t,Q{m)) = 
= 1 + "^^=1 ^^^^ Ci{Q{in)) = 1 for i = 1, . . . , m — 1. 

2.4 Certain bundles on Segre varieties 

Let nil, • • • 5 TTT'd > 2 be given integers with d > 1 and denote m = (m-i, . . . , m^) 
Consider the Segre variety S(m) := P(C"'i) x ... x P(C'"'^). Let VTi : S(m) - 
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') be the projection on the i-th component. Then ir* H{mi),TT*Q{mi) are the 
pull back of the bundles H{mi),Q{mi) on P(C™'') to S(m) respectively. Denote 
ti = ci{'K*H{mi)). The cohomology ring //*(S(m)) is generated by ti, . . . ,td with 
the relations i^' = 0, that is H*{T.{m)) ~ C[ti, . . . \ . . . ,1"^") and in the 

following we interpretate ti just as variables. Correspondingly, the Chern polynomial 
of a vector bundle E over S(m) can be written as C{t,E) = C{t, . . . ,t,E) where 
C{ti, . . . ,td,E) = Cati^ ■ ■ ■ t'^'', a = («!, . . . , Qfrf) and Ca G Z. The i-th Chern 
class Ci{E) coincides with the degree i summand ^\a\=i'^»^i^ ■ ■ •^d'*' where |a| = 
Yli=i'^i- Since we have t™"' = 0, note that the top Chern class coincides with 

Cmi-l,...,md-l 

Let R{i,ni) and i?(m) be the following vector bundles on $](m) 

R{i, m) = {(g)j(,[a]\{i}T^jH{mj)) TT*Q{mi), R{m) = e,;e[d]i?(i, m). (2.2) 

Note that rankii(i,m) = mj — 1 and rank ii(m) = dimS(m). From the Euler 
sequence 

0^{(^j^ld]\{i}-Jr*H{mj)) 7r*T{mi)^{(g)j^[a]\{i}T^*jH{mj)) C™>^i?(i, m)^0 
and from Whitney's formula yield 



+ 2^ke[d]\{i} 

C(t,i?(m))= J]C7(t,ii(i,m)). (2.4) 

In what follows we need to compute the top Chern class of i?(m). Since 
rank i?(m) = dimS(m), and S(m) is a manifold, it follows that the top Chern class 
of i?(m) is of the form c(m) ni6[d] t^e coefficient of idimS(m) -^^ (j^^^ R{m)). 

By abuse of language the coefficient c(m), which is a positive integer, is called also 
the top Chern class of R{m). 

Lemma 1 Let ii(m) he the vector bundle on the Segre variety S(m) given by 
( |2.2[ ). Then the top Chern class o/M(m), c(m), is the coefficient of the monomial 
Y\ie[d] t^^~^ in the polynomial 

rrii-l 
i(i[d] j=0 fce[d]\{j} 

Proof. Call ti = Eke[d]\{i} ^k- 

Write the contribution of the denominator of (|2.3l) as 



=^ = y'(ti - ii) 



expand the numerator of (2.3) as 



1+ E " =E -)'•. 
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Putting together, the summand of total degree — 1 in (2.3) is 

nii — 1 



which is equal to 



rrii-l 
j=0 

In conclusion, the top Chern class of R{i, m) is given by the coefficient of total 
degree - 1 in the polynomial J2T=o^i^ke\d]\{i} tkyt'l''~^~^{tj - tj)\ The iden- 



tity (2.4) is equivalent to the claim. □ 



Note that the polynomial (2.5) is equal to the polynomial given in (1.3). 



2.5 Bertini's type theorem 

Let M be a compact complex manifold and E a holomorphic bundle on M. A 
holomorphic section a of -E on open U <Z E \s a, holomorphic map a : U ^ E, 
where E is viewed as a complex manifold. Specifically, let Ui,i G [N] be the finite 
cover of M such that the bundle E restricted to Uj is C/,- x with the standard 



basis [ei_j, . . . ,ek^i], as in ^2.2, Then a{C) = J2j=i ^j,ii0^j,i for G ?7 n Ui. where 
^j,i{C)d ^ W\ ^I's analytic on C/n C/j. a is called a global section if [/ = M. Denote 
by 'B9{E) the linear space of global sections on E. A subspace V C H'^(£^) is said 
to generate E if V((^), the values of all section in V at each C G M is equal to Eq. 

The following proposition is a generalization of the classical Bertini's theorem 
in algebraic geometry, and it is a standard consequence of the Generic Smoothness 
Theorem. For the convenience of the reader we state and give a short proof of this 
proposition. 

Theorem 2 ("Bertini's type" theorem) Let E he a vector bundle on M. 
Let V C H'^(ii^) he a subspace which generates E. Then 

• if rank E > dim M for the general o" G V the zero locus of a is empty. 

• if rank E < dim M for the general cj G V the zero locus of a is either smooth 
of codimension rank E or it is empty. 

• i/rank E = dimM = n the zero locus of the general a £ V consists of Cn{E) 
simple points, where Cn is the n-th Chern class, (the top Chern class) and we 
identify H^'^{M) = Z. 

Proof. We identify the vector bundle E with its locally free sheaf of sections, 
see [6j B.3. We have the projection E-^X, where the fiber vr~^(C) is isomorphic 
to the vector space S^. Let T, C E he the zero section. By the assumption we have 
a natural projection of maximal rank 

M X \-^E. 

Let Z = p^^(S), then Z is isomorphic to the variety {(C, <7) £ E x V|(t(C) = 0} and 
it has dimension equal to dim M + dim V — rank E. Consider the natural projection 
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Z— ^V, now Vo" E V the fiber q~^(a) is naturally isomorphic to the zero locus of a. 
We have two cases. If q is dominant (namely the image of q is dense) then by the 
Generic Smoothness Theorem (^lOj Corol. Ill 10.7) q~^{cr) is smooth of dimension 
dim X — rank E for general a. 

If q is not dominant (and this always happens in the case rank E > dim M) then 
q~^{a) is empty for general a. This concludes the proof of the first two parts. The 
third part follows by [6] example 3.2.16. □ 



3 The number of singular vector tuples of a generic ten- 
sor 

In this section we compute the number of singular vector tuples of a general tensor 
TeC"". As pointed in ^ a singular value tuple is viewed as a point ([xi], . . . , [x^]) G 
S(m). With a slight abuse of notation we identify a vector Xj G C"^ \ {0} with the 
line [xj] in C™^ , which is a point in the projective space 



Lemma 3 Let R{i,ni),i £ [d] and R{m) be the vector bundles over the Segre 
variety S(m) defined in [2.4 Denote by II^(R(f , m)), i € [d] and I{^{R[ui)) the 
linear space of global section of R{i,in),i G [d] and R{in) respectively. Then the 
following conditions hold. 

1. For each i G [d] there exists a monomorphism Li : C"^ — )• Il'^(ii(i, m)) such 
that Li{C"^) generates i?(i,m) (see {2.5). 

2. L = (Li, . . . , Ld) is a monomorphism of the direct sum of d copies o/ 
(e'^C"') to HO(i?(m)). 

5. (5 : ^ e'^C"' be the diagonal map 5{T) = {T,...,T). Then L o 6 is a 
monomorphism o/C™ to Il'^(i?(m)). Furthermore Lo5{C"^) generates i?(m). 

Proof. Let T G C'" and uj G C"'^ \ {0},i G [d]. For x G C™' let [[x]] be the 
corresponding element in the quotient space C'"'/[uj] and consider global 
section of the hyperplane line bundle H{mj). View (Xjg[(^]\|j}.Uj)(g)[[x]] as an element 
in the fiber i?(z,m)(u^^ u^). Then 

Li(r)((ui, . . . , Wd)) ■■= ® [['^ ^ ®ieM\{i}]]- (3-1) 

It is straightforward to check that Li{T) is a global section of i?(i,m). 

Assume T / 0. Then there exist vj G C^^ ,j G [d] such that T x (E)je[d]Vj / 0. 
Hence x := T x (S'j^[d]\{i} G C™"' \ {0}. Let = Vj for j ^ i. Choose Uj G 
FiC""^) \ {[x]}. Then L(®jg[^] Vj)((ui, . . . , u^)) / 0. Hence Li is injective. 

Let Vj = ||uj||~^u* for j G [d] \ {i} and Vj = x. Then 

^i(%e[d]Vj)((ui,...,Ud)) = ( Yl Ui)®[W]- 

jm\{i} 

Hence L(C'") generates R{i,m.). 

Define L((71, . . . , 7d))((ui, . . . , u^)) = ®i(z[d]Li{{Ti, 7d))((ui, . . . , u^)). Then 
L((71, . . . ,Td)) G H'^(i?(m)). Clearly L is a monomorphism. 
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It is left to show that LoS(C"^) generates R{m) . By changing bases in C™^ , • • • , C"*'^ 
we may assume with a loss of generality that Uj = (1,0, .. . ,0)"'' G C'"' for i € [d]. 
Then point each [[xj]] G C"^^ /[ui] can be chosen of the form x, = (0, X2,i, ■ ■ ■ , Xmi,i)~^ ■ 
Choose T = [tji,...,jj as follows. = 1- For a fixed i G [d] let = 

Xj.^i for ji >2. All other entries of T are zero. Clearly 

Li(r)((ui, . . . , Urf)) = (%G[d]\{i}Uj) [[xj]], ie [d]. 

Hence L o (5(C™) generates i?(m). □ 



It follow from Kiinneth formula(|12j) the equalities 
Assume that (xi, , 



F°(fl(m)). 



(3.2) 



Xd) G S(m) is a a singular vector tuple of T G C™, i.e. 
(1.2) holds. We say that (xi, . . . ,x^) corresponds to zero or nonzero singular value 
if either Hjeld] nonzero respectively. For T G with a real singular 

vector tuple (xi, . . . ,Xrf) G SiR(m) the condition Tljelrf] ~ ^ implies that Aj = 

rx0jg[d]Xj 

^ : 0. Hence 
each A,- = 0. 



for each i. Indeed, since Xj G M™' \ {0} it follows from (1.2) that Aj = 
for each i G [d]. Thus Afc = for some k G [d] yields that T x '^j^[d]Xj 



However, this observation is not valid for complex tensors, already in the case 
of complex valued matrices {d = 2), see an example below. It is straightforward 
to see that a singular value pair (xi,X2) of A G C™'^^™'^ is given by the following 
conditions 



Aixi, A xi = A2X2, 
Consider the following simple example. 

A 



Xj G 



'^\{0},A,G 



1,2. 



(3.3) 



" 1 i ■ 




" 1 ■ 




" 1 " 


-i 1 







, X2 = 




i 



Then ^"^xi = X2, ^X2 = 0, i.e. Ai = 1, A2 = 0. 

A singular vector tuple (xi,...,x^) G S(m) is called a simple singular tuple 
of T G C", (simple), if there exist open neighborhoods Oi C S(m),02 C of 
(xi,...,X(i) and T respectively, such that each 5 G O2 has exactly one singular 
vector tuple in Oi. 

Theorem 4 A general T G has exactly c(m), the coefficient of the mono- 



mial nj=i ^r*'^ polynomial (1.3), simple singular vector tuples which corre- 

spond to nonzero singular values. A general T G does not have a zero singular 
value. In particular, a general real valued tensor T G has at most c(m) real 
singular pairs, which are simple. 

Proof. Let T G and consider the section f := L o 5(T) G H°(i?(m)). We 
claim that (xi, . . . , x,;;) G S(m) is a zero of T if and only if (xi, . . . , x^) is a singular 
vector tuple corresponding to T. Furthermore, (xi, . . . , x^) G S(m) is a simple zero 
of T if and only if (xi, . . . ,Xrf) is a simple singular vector tuple corresponding to 
T. Indeed, suppose first that r((xi, . . . ,X(^)) = 0. Then Lj(T)((xi, . . . is a 
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zero vector in the fiber m)(xi,...,Xd)- Suppose first that T x (nje[d]\{i} ^j) 7^ 0- 
Then T x (nje[d]\{i} ■'^i) ~ ^^^^ ^^'^ some Aj / 0. Otherwise the previous equality 
hods with Aj = 0. Hence (xi, . . . ,Xrf) is a singular vector pair corresponding to T. 
Viceversa, it is straightforward to see that if (xi , . . . , x^) is a singular vector pair 
corresponding to T then the section T vanishes at (xi, . . . , x^) G S(m). 

Since Lo(5(C™) is a generating subspace of H'^(i?(m)) and rank R{m) = dimS(m), 



Bertini's type theorem, see ^ 2.5 , yields that for a general T G the section Lo5{T) 
has a finite number of simple zeros, and this number is equal to the top Chern class 
of i?(m). Lemma [s] yields that the top Chern class of i2(m) is c(m). 

It is straightforward to show that (xi, . . . ,X(j) G E(m) is a simple zero T" if and 
only if it is an isolated zero and there exist open neighborhoods 0\ C S(m),02 C 
of (xi, . . . jXf^) and T respectively, such that for each 5 G O2 the section S 
has exactly one singular vector tuple in Oi. Clearly this fact is equivalent to the 
assumption that (xi, . . . , x^) G S(m) is a simple singular vector tuple of T- 

It is left to show that a generic T G C™ does not have a zero singular value. 
Fix i G [d\ and consider the set of all T G which have a singular vector tuple 
(xi, . . . , X(i) G S(m) with Aj = 0. Let Tj := C"' be the trivial vector bundle of rank 
rrii over P(C'"'). Denote by T{i,m.) := vr*Tj the pull back of this trivial bundle to 
E(m). Let 

R{i,m)' := {^j^ia]\{i}T^jH{mj))0T{i,m.), i?i(m)' := {(Bje[d]\{i}R{j,m))(BR{i,uiy. 

Note that i?i(m)' is a vector bundle on S(m) of rank dimS(m) + 1. As above we 
can define a monomorphism L[ : — )• H'^(-R(i, m)') defined by the equality 

mT){{ui, . . . ,Ud)) := (0jg[rf]\|i|Uj) (g) (r X 0j-g[rf]\|i|Uj). 

Let Li = (Li, . . . ,Li_i,L'.,L,+i, . . . , L^) : ejeirfjC" ^ R^{Ri{my). We claim that 
Lio5{C"^) generates i?j(m). It is enough to consider the point (ui, . . . , u^) G S(m), 
where = (1, 0, . . . , 0)^ G for i G [d]. Let xj = (xij, . . . , Xm.jV G C™J for 
j G [d]. Choose T = [tji,...,j^] as follows. = a^i^j. For a fixed j G [d] let 

^i,...,i.fcj,i....,i = ^kj.j for /cj > 2. All other entries of T are zero. Then 

Lj(r)((ui, . . . , Ud)) = ((^fce[d]\{j}Ufe) (g) [[xj]], j G [d] \ {i}, 
Zi(r)((ui, . . . , Ud)) = {(E>ke[d]\{i}Uk) «) Xi. 

Hence Li o 5(C"^) generates i?j(m)'. Theorem [2] yields that for general T G the 
section Lj o S{T) has no zeros. That is, T does not have a singular vector tuple 



satisfying (1.2) with Aj = 0. Hence a general tensor T G does not have a zero 
singular value. 

In algebraic geometry the term general T means that T G \ X, where X is 
some algebraic variety, given as a zero set of a finite number of polynomial in the 
entries of T- A general real T G M"" does not lie in X. Hence a general T G M™ 
has exactly c(m) simple singular value tuples. Only some of those can be realized 
as points in Il]R(m). □ 

We first observe that Theorem |4] agrees with the standard theory of singular 
values for m X n real matrices. Let A G C™!^™^ ^^id Ui G C™'^™' (for i = 1, 2) be 
(complex) orthogonal matrices. Suppose that (3.3) holds. Let B = UiAUj . Then 
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{Uixi, U2X2) is a singular pair of B. Suppose that there exists C/i, f/2 such that B is 
a diagonal matrix such that the absolute values of its diagonal entries are positive 
and pairwise distinct. Then a straighforward calculation shows that B has exactly 
min(mi,m2) singular pairs corresponding to nonzero singular values. The singular 
value decomposition of ^ G ]^»n-ixm,2 implies that a general A € M''"i^''"2 has exactly 
min(mi, 1712) singular pairs corresponding to nonzero singular values. It is also easy 
to see that c{mi,m2) = min(mi,m2). 

We point out a matrix proof of Theorem [4] for d = 2. Let 0(m) C C"*^™ be 
variety of m x m orthogonal matrices and T)rn,n C C*"^" the linear subspace of all 
diagonal matrices. Consider the trilinear polynomial map {Ui, 0,1/2) '-^ Uj DU2 
from the 0(mi) x Bmi,m2 x 0(1712) to C'"^^™'^, ig not difficult to show that the 
image of this map is dense in C'^iX'^a^ since over the reals this map is onto. The 
above arguments give a simple proof of Theorem |4] for d = 2. 

We list for the convenience of the reader a few values c(m). First 

c(2, ...,2) = d! (3.4) 

d 

Second, we list at next page the first values in the case d = 3. ^From this table 
one sees that c(?tii, m2, 711-3) stabilizes for > mi + m2 — 1, the case when equality 
holds is called the boundary format case in the theory of hyperdeterminants ([Z])- 
It is the case where a "diagonal" naturally occurs, like in the following figure 




In d = 2 case, boundary format means square. 
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di 


,d2,d3 


c{di,d2, ds) 








2,2,2 


6 








2,2,n 


8 


n > 3 






2,3,3 


15 








2,3,4 


18 


n > 4 






2,4,4 


28 








2,A,n 


32 


n > 5 






2,5,5 


45 








2,5,n 


50 


n > 6 


2, 


m, 


m + 1 


2m2 








3,3,3 


37 








3,3,4 


55 








3, 3, n 


61 


n > 5 






3,4,4 


104 








3,4,5 


138 








3,4,n 


148 


n > 6 






3,5,5 


225 








3,5,6 


280 








3, 5, n 


295 


n > 7 


3, 


m, 


m + 2 


— 2m? + Im 








4,4,4 


240 








4,4,5 


380 








4,4,6 


460 








4, 4, n 


480 


n > 7 






4,5,5 


725 








4,5,6 


1030 








4,5,7 


1185 








4, 5,n 


1220 


n > 8 






5,5,5 


1621 








5,5,6 


2671 








5,5,7 


3461 








5,5,8 


3811 








5, 5, n 


3881 


n > 9 



4 Partially symmetric singular value tuples 



For an integer m > 2 let m 



xd 



(m, . . . ,m). Then T G F"^^ is called d-cube. 



d 

or simply a cube tensor. Denote by S'^(F™') C F™''" the subspace of symmetric 
tensors. For T S S'^(F™') it is natural to consider a singular tuple (1.2) where 
xi = . , 



= X |14l formula (7) with p = 2]. This is equivalent to the system 



r X 



.d-l 



X = Ax, X 7^ 0. 



(4.1) 



Here < 



X := X ( 



X. Furthermore, the contraction in (4.1 ) is on the last d—1 



d-l 



For d = 2 X is an eigenvector of the square matrix T. Hence for a d-cube tensor 



indices. The equation (4.1) makes sense for any cube tensor 7" G 
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(d > 3) X is referred as a nonlinear eigenvalue of T. Abusing slightly our notation we 
call (x, . . . , x) G S(m^'^) a symmetric singular tuple of T. (Note that if T G S'^(C'") 
then (x, . . . , x) is a proper symmetric singular tuple of T-) 

Let Sd-i{T) = be the symmetrization of a d-cube T = [ti^,...,i^] with 

respect to the last d — 1 indices 

b2,---Jd} = {*2v,«d} 

Herep(i2, . . . , id) is the number of multisets {j2, . . . , jd} which are equal to {z2) • • • , 'id}- 
(Note that ioi d = 2 si(T) = T.) It is straightforward to see that 

T X (gj'^-V = Sd-^i{T) y for all y. (4.3) 



Hence in (4.1) we can assume that T is symmetric with respect to the last d — 1 
indices. 

As for singular value tuples we view the eigenvectors of T as elements of P(C'"). 

X d 

It was shown by Cartwright and Sturmfels ^ that a general T G has exactly 

^'^~d-2~^ distinct eigenvectors. (This formula was conjectured in [IT]-) 

The aim of this section is to consider "partially symmetric singular vectors" 
and their numbers for a general tensor. This number will interpolate our formula 
c(m) for the number of singular tuples for a general T G and the number of 

X d 

eigenvalues of general T G given in [2j . 

Let d = ioi + ...+ cOp he a partition of d. So each is a positive integer. Let 
ujQ = tjiq = 0, ll> = (wi, . . . ,ujp) and denote by m(a;) the d-tuple 



miuj] = [m^, ... ,'m[, ... ,mp, ... ,771^) = {mi, . . . ,md). (4.4) 



Denote by S^(F) C F'"^'^'' the subspace of tensors which are partially symmetric 
with respect to the partition uj. That is the entries of T = G S^(F) are 

invariant, if we permute indices in the k — th group of indices [J2j=o^j] \ [X]j=o ^j] 
for G \p]. Note that S'^(F) = S"'(F'")) for p = 1 and S'^(F) = F'" for p = d. We 
call a; = (1, . . . , 1), i.e. p = d, the trivial partition. 

For simplicity of notation we let S'^ := S^(C). Assume that T G S'^. Consider 



a singular vector tuple (xi, . . . ,Xd) satisfying (1.2) and a;-symmetric conditions 



k k-1 



Xj = Yk for j G [^Wjm-] \ [^Wim-], k G [p]. (4.5) 



1=1 i=0 



We rewrite (1.2) for an cj-symmetric singular vector tuple (xi, . . . ,Xd) as follows. 
Define 

^lelp] (®"'"'"^yO := ®,eM\{i+Eto^..,m:}^i' k G [p]. (4.6) 

Hence our equations for an cu symmetric singular value tuple for T G is given 
by 

T X »/e[p](^"'-^'''yO = XkYk k £ [p]. (4.7) 
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In view of the definition of y/) we agree that the contraction on the 

left-hand side of (4.7) is done on all indices except the index 1 + Xlto^ ujim'-. As for 



the d-cube tensor the system (4.7) makes sense for any T G C'"^'^^. 

Let m' := (m'|, . . . , rrip). We call (yi,...,yp) G S(m') satisfying (4.7) as u- 



symmetric singular tuples of T G C"^^^\ We say that (yi, . . . ,yp) corresponds to 
a zero (nonzero) singular value if HiLi = (t^ 0)- 

The aim of this section to show that a general tensor T G S'*^ , hence a general 
tensor T G C"^^'*^^ has exactly c(m', u) cj-symmetric singular tuples, where c(m', cj) 

is the coefficient of the monomial Hieb] ^ homogeneous polynomial 

ieb] * * ie[p]\{i} 

The proof of this result is analogous to the proof of Theorem |4| so we point 
briefly the needed modifications. Let H{m'-) and Q{m'-) be the vector bundles 



defined in ^2.4 Let tTj be the projection of S(m') on the component ¥(C"^i). Then 
■K*H{m'j), Tr*Q{m'^ are the pull backs of the vector bundles H{m'j), Q{m^) to E(m') 
respectively. Clearly C{t,Tr* H{m'^) = 1 + ti and moreover C{t,®'^'K*H{m'j)) = 

1 + kti, where t^' = 0. Define the following vector bundles on S(m'). 

i?(i,m') := (»,e[p](»"^-^^'7r*F(m;.))) ^<Q(m9, i G [p], 

i?(m') :=e,e[p]i2(i,m'). (4.9) 

Note that rank i?(i, m') = — 1 and rank i?(m') = dimE(m'). As in the proof of 
Lemma[l]we deduce that the top Chern class of R{i, m') is given by the polynomial 

m'-l 

5;(j;(a;fc-fe)tfe)^'C^''"'), i^[pl (4.10) 

i=0 fc6[p] 

where we assume the relations t™' = for i G [p]. Use Whitney's formula to deduce 
that the top Chern class of R{m') is c(m',a;). 

We next observe that we can view S(m') as a submanifold by using the imbed- 
ding T] : S(m') — S(m(a;)). Here r/((yi, . . . ,yp)) = (xi, . . . ,X(^) where we assume 



the relations (4.5). It now follows that R{i,in') is a pull back i2(j, m), where 
j = 1 + X^fclg ^kfn'k- From the results of ^ in particular Lemma [sj we deduce that 
there exists a monomorphism Li : C^^^^ — t- }i^{R{i,m')). Furthermore Li(C"^^^^) 
generates R{i,ui'). 

Let L = (Li, ...,Lp): e.eipjC'"^^) ^ R^{R{m')). Then L o (5(C'"(^)) generates 
HO(i?(m')). 

Theorem 5 Let d > 3 be a an integer and assume that uj = (t^i, . . . ,ojp) is a 
partition of d. Let m(a;) he defined by (4.6). Denote by C C'"^'*^^ the suhspace 



of tensors partially symmetric with respect to u. A general T G S has exactly 



c(m',a;), the coefficient of the monomial Y[^=iti * in the polynomial (4.8), simple 
u -symmetric singular vector tuples which correspond to nonzero singular values. A 
general T G S'*' does not have a zero singular value. In particular, a general real 
valued tensor T G has at most c(m',a;) real singular pairs, which are simple. 
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Proof. The observations given before the statement of the theorem yield all the 
results for a general T G C'"^'*^^. Observe next that Lo5{S^) generates i?(m'). It is 
enough to show that Lo6{S^) generates the fiber R[m') over the point (ui, . . . , Up), 
where = (1,0, . . . ,0)^ G C< for i G [v\. Each [[xi]] G C</[ui] can be chosen of 
the form Xj = (0, X2,i, x^' j)"^. Choose T = as follows. = 1- For 

a fixed i G [d] let = for > 2, where q{i) = 1 + Efc=Wfc- 

Clearly 

Li(r)((ui, . . . ,Up)) = (%6[p](0'^^-"^'^uj) » [[xi]], i G [p]. 

Hence Lo 6{S^) generates R{m'). Bertini's theorem yields that a general T G S'^ 
has exactly c{m!,u) simple o^-symmetric singular vector tuples. The proof that a 
general T G S'^ does not have a zero singular value is analogous to the proof given 
in Theorem m □ 



Remark 6 In the special case cj = (1, 1, . . . , 1) we have c{m' = c(m') and 
Theorem^ reduces to Theorem J^. In the case u = (d) we have c{m,u) = — " 
and Theorem reduces to the results in J^. This last reduction was performed 
already in flW . 

Lemma 7 In the case cj = (d — 1, 1) we have 



mi-l m2-l / .\ 

c((mi,m2),(d-l,l))= ^ [])id-2yid-iy-^. 

i=0 j=0 ^-^^ 



If TT^i < "^2 we have c((mi, 7712), {d — 1, 1)) = ^^'^ 2d- 4^ — ~ 

If mi = m2 + 1 we have c((mi, 7712), {d — 1, 1)) = ^^'^ -m-I — ^ — [d — 

Note that in the 3x3x3 case our formula gives c(3, 3, 3) = c((3, 3, 3), (1, 1, 1)) = 37, 
while Cartwright-Sturmfels formula gives 7, counting indeed triples (x, x, x). Note 
also that the number 37 was computed, in a similar setting, in [16j. 

Moreover the previous lemma gives c((3, 3), (2, 1)) = 13 for the singular vectors 
of the form (x, x, y). 

It is instructive to list all the singular vectors for the diagonal tensor T = 

[A. . A. . 1 fi (p3x3x3 
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(a;0,xl,x2)(y0,yl,y2)(z0,zl,z2) 


singular value 




(1,0,0)(1,0,0) 


(1,0,0) 


1 




(0,1,0)(0,1,0) 


(0,1,0) 


1 




(0,0,1)(0,0,1) 


(0,0,1) 


1 




(1,1,0)(1,1,0) 


(1,1,0) 


1 




(1,0,1)(1,0,1) 


(1,0,1) 


1 




(0,1,1)(0,1,1) 


(0,1,1) 


1 




(1,1,1)(1,1,1) 


(1,1,1) 


1 




(1,1,0)(1,-1,0) 


(1,-1,0) 


1 


3 permutations 


(1,0,1)(1,0,-1) 


(1,0,-1) 


1 


3 permutations 


(0,1,1)(0,1,-1) 


(0,1,-1) 


1 


3 permutations 


(1,1,1)(1,1-1) 


(1,1,-1) 


1 


3 permutations 


(1,1,1)(1,-1,1) 


(1,-1,1) 


1 


3 permutations 


(1,1,1)(-1,1,1) 


(-1,1,1) 


1 


3 permutations 


(1,0,0)(0,1,0) 


(0,0,1) 





6 permutations 


(1,1,-1)(1,-1,1 


)(-l,l,l) 


-1 


6 permutations 



In this list the first 7 singular vectors have equal entries and they are the one 
counted by the formula in [2] . 

The first 7 + 6 = 13 singular vectors have the form (x,x, y). Any singular vector 
of this form gives 3 singular vectors (x, x, y), (x, y,x), (y,x,x). 

Note that six singular vectors have zero singular value, but this does not cor- 
respond to the general case, indeed for a general tensor all 37 singular vectors 
correspond to nonzero singular value. 

In the case of 4 x 4 x 4 tensors, the diagonal tensor has 156 singular vectors 
corresponding to nonzero singular value and infinitely many singular vectors corre- 
sponding to zero singular vectors. These infinitely many singular vectors fill exactly 
36 projective lines in the Segre variety P(C^) x P(C'^) x P(C^), which "count" in this 
case for the remaining 240 — 156 = 84 singular vectors. 



5 A homogeneous pencil eigenvalue problem 



For X = {xi,...,XmV G denote x°(d-i) ._ (xf-\ . . . , x^^y . Let T G C™""". 
The eigenvalues of T satisfying (4.1) are called the i?-eigenvalues in |20j . The 
homogeneous eigenvalue problem introduced in [15] , [13] and [19] , sometimes referred 
as A^-eigenvalues, is 

r X ®'^-ix = Ax°('^-i), x/0. (5.1) 

X d 

Let S G C™ . Then a generalized d — 1 pencil eigenvalue problem is 

r X ^'^"ix = A5 X ®^"^x. (5.2) 

For d = 2 the above homogeneous system is the standard eigenvalue problem for a 
pencil of matrices T, S. 

A tensor S is called singular if the system 

S X (gj'^^^x = (5.3) 
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has a nontrivial solution. Otherwise S is called nonsingular. It is very easy to 
give an example of a symmetric nonsingular S [5]. Let wi,...,Wm be linearly 
independent in C". Then S = Yl'iLi^'^'^i is nonsingular. The set of singular 
tensors in C™^'' is a given by the zero set of some multidimensional resultant [71 
Chapter 13]. It can be obtained by using the elimination of variables. Let us denote 
by res 5 the multidimensional resultant corresponding to the system ( |5.3| ), which 
is a homogeneous polynomial in the entries of S of degree ii{m,d) = m{d — 1)"*"^ 

X d 

[71 Proposition 1.1]. That is, res S is an irreducible polynomial in C™ such that 



the system (5.3) as a nonzero solution if and only if res 5 = 0. Furthermore for a 



general point S G Z(res ), i.e. the zero set of the polynomial res , the system (5.3) 



has exactly one simple solution in P(C™'). The eigenvalue problem (5.2) consists of 
two steps. First find all A satisfying res {XS — T) = 0. Clearly res {\S — T) is a 
polynomial in A of degree n{m, d) at most. (It is possible that this polynomial in 
A is a zero polynomial. This is the case where there exists a nontrivial solution to 
the system S x = T x = 0.) After then one needs to find the nonzero 
solutions of the system {XS — T) (^'^~^ x = which are viewed as eigenvectors 
in P(C™). Assume that S is nonsingular. Then res {XS — T) = res {S)X^^^''^^ + 
polynomial in A of degree fi{m,d) — 1 at most. We show below a result known to 
the experts, that for general S, T each eigenvalue A the system {XS — T) x = 
has exactly one corresponding eigenvector in P(C™'). We outline a short proof of the 
following known theorem which basically uses only the existence of the resultant for 



the system (5.3). For an identity tensor S. i.e. (5.1), see [T9] . 



Theorem 8 Let 5, T € C" and assume thatS is nonsingular. Themes {XS— 
T) is a polynomial in X of degree m{d — 1)^^^ . For a generic S and T to each 



eigenvalue X of the pencil (5.1) corresponds one eigenvector in P(C'' 



Proof. Consider the space F{C^) x P(C'"'"' x C"""") x P(C™) with the local 
coordinates {{u,v), {S,T),x). Consider the system of m-equation homogenous in 
{u,v), (5,T),x given by 

{uS - vT) X o'^-^x = 0. (5.4) 

The existence of the multidimensional resultant is equivalent to the assumption 
that the above variety V{m^ d) is an irreducible variety of dimension 2m'^ — 1 in 
P(C2) X P(C™''' X C"""") X P(C™). So it is enough to find a good point {So,%) 
such that it has exactly fi{m,d) = m{d — 1)™~^ smooth point {{ui,Vi), (5o,7o),Xj) 
in V{m, d). 

To do that we assume that 5o, To are in fact "matrix type" tensors. 

SqX®'^-^^ = ^x°('^-i), Tox^'^-ix = Sx°('^-i), A = [a,,j\,B = h,j\ G C™^™. (5.5) 

to be specific we can choose A = I and i? as a cyclic permutation matrix B{xi , . . . , Xm) 
= {x2, • . . , Xm, xi)~^ . Then B has m distinct eigenvalues, the m — th roots of unity. 
X is an eigenvector of ( 5.5 ) if and only if x°('^~^) is an eigenvector of B. Fix an eigen- 



value of B. One can fix xi = 1. Then we have exactly {d — l)"*"! eigenvectors in 
P(C'") corresponding to each eigenvalue A of B. So altogether we have m{d— l)™-i 
distinct eigenvectors. It is left to show that each point ((u,, Vj), (5o, To), Xj) is a 
simple point of V{m, d). For that we need to show that the Jacobian of the system 



(5.4) at each point has rank m, the maximal possible rank, at {{ui,Vi), (5o,To),Xi 



17 



For that we assume that Uj = Xi,Vi = l,xi = 1. This easily fohows from the fact 
that each eigenvalue of S is a simple eigenvalue. Hence the projection of V{'m, d) 
on P(C™'^ X C"""") is m{d - 1)™"^ valued. 

m—l 



Note that in this example each eigenvalue A of (5.5 ) is of multiplicity (d — l) 
It is left to show that when we consider the pairs So , T where T varies in the neigh- 
borhood of To we obtain m{d — different eigenvalues. Since the Jacobian of 



the system (5.5 ) has rank m at each eigenvalue Aj = ^ and the corresponding eigen- 
vector Xj, one has a simple variation formula for each 5Xi using the implicit function 
theorem. Fix xi = 1 and denote F(x, A, T) = (Fi, . . . , Fm) := (A5o — T) x (gj'^^^x. 
Thus we have the system of m equations F(x, A) = in m variables X2, ■ ■ ■ , Xm, A. 
We let T = To + tTi and we want to find the first term of Xi{t) = Aj -|- Oit + 0{t^). 
We also assume that Xj(t) = Xj -|- tyi + O(t^), where = (0, 7/2,1, • • • , ym,i)~^ ■ Hence 
the first order computations yield the equations 

Ti X O'^-^Xj + ToXZi = aiSo x ^"'"^Xj + XiS x Zj, (5.6) 

Zi= ®^'"^Xi ® Yi (g)'^-^-^ Xj. 

je[d-i] 

Let w = {wi, . . . ,Wm)~^ be the left eigenvector of B corresponding to Xi, i.e. 
B = AjW^ normalized by the condition w'''(5o x (8)'^~^Xj) = (Sq x (8)'^~-^Xj) x w = 



1. Contracting both sides of (5.6) with the vector w we obtain 



ai = Ti X (w® (^j'^-^Xi)). (5.7) 

It is straightforward to show that ai, . . . , Oim{d-i)"^-^ pairwise distinct for a 

general Ti . □ 

The proof of Theorem |8] yields. 

X d 

Corollary 9 Let T G C™ ) be a general tensor. Then the homogeneous eigen- 
value problem (5.1) has exactly m{d — 1)"^~^ distinct eigenvectors in P(C™) which 



correspond to distinct eigenvalues. 

We close this section with an heuristic argument which shows that a generic 
pencil (5, T) G P(C'" x C™ ) has /i(rn,, d) = m{d — 1)"^-! distinct eigenvalues in 

X d " 

P(C'"). Let S G C" be nonsingular Then S induces a linear map S from the line 
bundle ®'^-^T{m) to the trivial bundle over P(C'") by (gj'^-^x ^ S x (^'^-^x. 
Then we have an exact sequence of line bundles 

^ ®'^-'^T{m) ^ Qm,d 

where Qm,d = C"" /{S{(S)'^-^T{m))). The Chern polynomial oiQm,d is 1 + Ei^"i^(f^- 



l)*t*a*. A similar computation for finding the number of eigenvectors of (4.1 ) shows 
that the number of eigenvalues of (5.2) is the coefficient of t^~^ in the polynomial 

^t— Here ii = ii = (d — l)ti. Hence the coefficient of t"^~^ is • 
The calculus interpretation of this formula is the derivative of t"^ at t = d—l, which 
gives the value of the coefficient m{d — 
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6 On uniqueness of best approximations 



Let (•,•), II • II be the standard inner product and the corresponding EucHdean norm 
on M". Let C C be a given nonempty closed set. For each x G R" we consider 
the function dist(x, C) := min{||x — y||, y G C}. A point y* G C is called a best 
approximation of x if ||x — y*|| = dist(x, C). The following result is well known and 
we bring its short proof for completeness. 

Lemma 10 Let C C M" he a given closed set. Let U C M" he a subspace 
with dimU G [n] and such that U is not contained in C. Let (i(x),x G \J he the 
restriction o/dist(-,(7) to U. 

• (i) The function dist(-,C) is Lipschitz with constant constant 1: 

|dist(x, C) - dist(z, C)| < ||x - z|| for all x, z G M". (6.1) 



• (ii) The function d{-) is differentiahle a.e. in U. 

• (Hi) Let X G U \ C and assume that d{-) is differentiahle at x. Denote the 
differential as dd{x) which is viewed as linear functional on U. Let y* G C 6e 
a best approximation to x. Then 

ad(x)(u) = (u, } (x - y*)) for each u G U. (6.2) 
dist(x, 6) 

If z* is another best approximation to x then z* — y* is orthogonal to U. 
Proof. Assume that dist(x, C) = ||x — y*||. Then 

dist(z, C) < ||z - y*|| for each z G M". (6.3) 
Suppose next that dist(z, C) = ||z — y||,y G C. Hence 

— ||x— z|| < ||x— y*|| — ||z — y*|| < dist(x, C)— dist(z, C) < ||x— y|| — ||z — y|| < ||x — z||. 



This proves (6.1) and (i). We get that d{-) is also Lipschitz on U. Rademacher's 
theorem yields that d{-) is differentiahle a.e., so proving (ii). In order to prove (iii), 
fix u G U. Then 

dist(x + tu, C) = dist(x, C) + tdd{x){u) + to{\t\). 



(6.3) yields the inequality 

dist(x + tu,C) < ||x + tu-y*|| = ||x - y* II + t{u, ^.^^^^ (x - y*)) + O(t') 
Compare this inequality with the previous equality to deduce that 



for all t G M. This implies (6.2). If z* another best approximation to x then (6.2) 
yields that z* — y* is orthogonal to U. □ 



19 



Corollary 11 Let C C M" be a given closed set. 

• (i) The function dist{x, C) has differential a. e. inM."'. 

• (ii) Let X G M" \ C and assume that dist(-,C) is differentiable at x. Then x 
has a unique best approximation y(x) E C. Furthermore 

9dist(x, C)(u) = (u, — L^(x - y(x))) for each u G R^. (6.4) 
dist(x, G) 

/n particular, a. a. x G M" /laue a unique best approximation y(x) G C 



Proof. Choose U = M", so (i(-) = dist(-,C) is differentiable a.e. by Lemma 10 



(ii). In order to prove (ii), assume that y* and z* are best approximations of x. 
Then z* — y* is orthogonal to M" by Lemma 10 (iii). So z* — y* = 0. Furthermore 
dell) holds. □ 



7 Best rank one approximations of d-mode tensors 

On define an inner product and its corresponding Hilbert-Schmidt norm {T, S) : = 
TxS, \\T\\ = ^J{T,T). We first present some known results of best rank one ap- 
proximations of real tensors. In this section we assume that F = M and T G W^. 
Let S™""^ C be the m — 1-dimensional sphere ||x|| = 1. Denote by S(m) the 
(i-product of the spheres S™^^^ x . . . x S™'*^"'^. Let (xi, . . . , x^) G S(m) and associate 
with (xi, . . . , x^) the d one dimensional subspaces Uj = span(xi), i G [d\. Note that 

II ®ie[d] Xill = W ||xi|| = 1. 

The projection -P® jg^^jUi {T) of T onto the one dimensional subspace U := ®i^[d\^i C 
®ie[(i]IK™% is given by 

/r(xi, . . . ,Xd) Xi,/r(xi,. . . ,Xrf) := (T, ®ig[rf]Xi), (xi, . . . ,Xd) G S(m). (7.1) 

Let -P((g).gj^jUi)-'- ("T") be the orthogonal projection of T onto the orthogonal comple- 
ment of (8'jg[rf]Ui. The Pythagorean identity yields 

IITf = l|p«.,„,^(r)f + l|P(«^^j^,u,)4'r)f. (7.2) 

With this notation, the best rank one approximation of T from S(m) is given by 

min min ||T — a xJI. 

(xi,...,Xd)eS(m) a6M ' ' 

Observing that 

mm||r-a®ie[d]Xi|| = HT - P^^.g^u, ('^)ll = ll^(®,eMU,)^(T)||, 

it follows that the best rank one approximation is obtained by the minimization of 
ll"^{'Xiie[d]Ui)^('^-^ll" view of (7.2) we deduce that best rank one approximation is 
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obtained by the maximization of ||-P(g)ig[^]u. C^)!! ^^^^ finally, using (7.1), it follows 
that the best rank one approximation is given by 

(Ti(r) := max /r(xi, . . . , x^). (7.3) 

(xi,...,Xd)eS(m) 

As in the matrix case cri(T) is called in [1^ the spectral norm. Furthermore it is 
shown in [11] that the computation of o"i(T) in general is NP-hard for d > 2. 

We will make use of the following result of [2], where we present the proof for 
completeness. 



Lemma 12 For T G W^, the critical points of /|s(m)7 defined in (7.1), are 
singular value tuples satisfying 

T X ((»je[d]\{i}Xj) = Axi for all i G [d], (xi, . . . ,Xrf) G S(m). (7.4) 

Proof. We need to find the critical points of (T, ®j<^\d\^i) where (xi, . . . , x^) G 
S(m). Using Lagrange multipliers we consider the auxiliary function 

c/(xi,...,Xd) := (T,(»je[d]Xj) - ^ Ajxjx^. 

The critical points of g then satisfy 

T X (0j-g[^]\|j|Xj) = AiXi, i G [d], 



and hence {T = AjX^ Xj = Aj for all i G [d\ which implies (7.4). □ 



Observe next that (xi , . . . , x^) satisfies ( 7.4 ) if and only if the vectors (±xi , . . . , ix^) 



satisfy (7.4). In particular, we could choose the signs in (±xi, . . . ,±Xrf) such that 
each corresponding A is nonnegative and then these A can be interpreted as the 
singular values of T. The maximal singular value of T is denoted by (Ti(T) and 



is given by (7.3). Note that to each nonnegative singular value there are at least 
2<i-i singular vector tuples of the form (±xi, . . . ,ibxrf). So it is more natural to 
view the singular vector tuples (xi, . . . ,Xd) as points in the real projective Segre 
variety S]j(m). Furthermore, the projection of T on the one dimensional sub- 
space spanned by Cg'igfd] (±Xj), where (xi,...,Xd) G S(m), is equal to one vector 

(T X (^ig[d]Xi) Xj. 

Theorem 13 There exists an variety V C such that for each T G \ V 
the best rank one approximation is unique. 

Proof. Theorem |4] implies the existence of a variety Vq C C'" such that each 



T G C'" \ Vq has exactly c(m) distinct singular value tuples. Lemma 12 implies 



for any T G M"^ the best rank approximation is of the form Ap ^j£[d\ ^j,p: Here 



(xi^p, . . . , Xd^p) G S(m) satisfies (7.4) with A = Ap > and Xj = Xj^p,j G [d]. 
Furthermore Ap = ai{T), where ci(7') is the maximum of all Ap. Assume that 
T G M"" \ Vq. Then the number of rank one tensors '^je[d]^j,p is at most c(m). 
Hence there exists a real subvariety V C M"^, which contains Vq D M"^ such there 
are exactly k > 1 different rank one tensor '^je[d]^j,p such that Ap = (Ti(7~). This 
implies that that for each T G M"" \ 1^ we have exactly k best rank one approxi- 



mation. Recall now Corollary 11 for C being the variety of all tensors of rank one 
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plus the zero tensor. This Corollary yields that for almost all T the best rank one 
approximation is unique. Hence k = 1. □ 



Let u = (wi, . . . , Wp) be a partition of d. For T G S^(M) it is natural to 
consider a best rank one approximation to T of the form it J^^gj^j (8)'^»Xj where 
(xi, . . . ,Xp) G S]R(m'). We call such an approximation a best 4J-symmetric rank 
one approximation. (The factor it is needed only if each is even.) As in the case 
T G a best a;-symmetric rank one approximation of 7" G S'^ (M) is a solution to 
the following maximum problem. 

max IT X (gjjpr^i (g)'^' Xjl. (7.5) 

(xi,...,Xp)GS(m') 

As before, the critical points of the functions ±7" x (8>ig[p] Xj on S(m') satisfy 

T X xj- = Axi, i G [p], (xi, . . . , Xp) G S(m'). (7.6) 

The best a;-symmetric rank one approximation corresponds to all A for which |A| 



has a maximal possible value. Combine the arguments of the proof of Theorem 13 
with Theorem [5] to obtain. 

Proposition 14 There exists a variety U C S'^(M) such that for each T G 
S'*^(M) \ U the best rank one u: -symmetric approximation is unique. 

Assume that <8>jg[d]yj G M™^^) is a best rank one approximation to a tensor 
T G S^(M). It is not obvious a priori that (gijgjdjyj is cj-symmetric. However, the 
following result is obvious. 

'^je[d]y<j{j) is best rank one approximation of T G S'^(]R) (7.7) 
for each permutation cr : [d] — )• [d] which preserves S'^(M). 



Lemma 15 For a. a. T G S'^(M) there exists a unique rank one tensor (Si j^[d]yj G 
I^rn(^) such that all best rank one approximations ofT are of the form (7.7). 

To prove this lemma we need an auxiliary lemma. 

X d 

Lemma 16 Let 'Sjt=[d]Xj,'S'jt=[d]yj G IR" • Assume that 

{^jedXj,0'^u) = i^jedYj^^'^u) Vu G M". (7.8) 

Then there exists a permutation o" : [d] — t- [d] such that 'Sj£[d]yj = '^je[d]^a{j)- 
Proof. Note that the condition (17. 8|) is equivalent to the equality 



Yl u^Xj = Yl u^yj Vu G M". (7.9) 

If (g)jgrfXj = then Hje^d] ^~^yj ~ ^ ^- Hence = for some j, so 

'Sj(z[d]yj = 'Sj(z[d]^j = 0. So we assume that '^je[d]^jj'i^je[d]yj both nonzero. 
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We now prove the lemma by induction. For d = 1 the lemma is trivial. So assume 
that the lemma hold of d = k. Let d = k + 1. Assume that u G span(xfc-(_i)-'". Then 

0. Hence span(xfc+i)-'- C Ujg[fc+i]span(yj). There- 



g yields that Uj&[d]'^ yj 

fore there exists j ^ [k + 1] such that span(xfc-|_i)- 
for some t G M \ {0}. Hence there exist zi, 
a' : [k + 1] ^ [k + 1] such that (S'jg[fc_|_iiz^(j) 
Hence ^j(z[k+i]^j and ^j^[k+i]'^j 



span(yj)-^. So yj = txk+i 
. , Z(i+i G and a permutation 



'je[fc+i]yi where z^+i 
and 



Zj satisfy (7.9). Therefore ®j^[k]^j and ^jef/tjZj 



isfy (7.9). Use the induction hypotesis to deduce the lemma. 



Xfc+l. 

sat- 

□ 



Proof of Lemma 15 We use Corollary 11 as follows. Let 



and assume that C is the set of all rank one tensors plus the zero tensor. We let 
U := S'^(M). Assume that d(-) is differentiable at T G S'*^(M) \ C. Suppose that 
'^je[d]yjy'^je[d]'^j SI'S best rank approximations of T- So 



(7i{T) = II (S)je[d] yj\ 



n iiy.i 

i6[d] 



n 



> 0. 



Without loss of generality we may assume that 

llyill = ll^ill = o-i(T)^ Vi G [d] 



(7.10) 



Corollary 11 yields that 



[fi^ielp] ^ ^i,'^je[d]yj 
The above equality is equivalent to 



%e[d]Zi> = Vui G M™'' i G [p]. 



n n "^y" 



i G [p], 



(7.11) 



where = and Oj = X]fc=o for all i G [p]. 

Suppose first that p = 1, i.e. S^(M) is the set of all symmetric tensors in 
(Note that d 
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a;i.) Then Lemma 
for some permutation a : [d] ^ [d]. This proves our lemma for p 



and (7.11) yields that 



1. 



«'je[d]y<7(i) 



Assume now that p > 1. Fix A; G [p]. Fix Uj G i G [p] \ {/c}. Let 

^^•= n n 

ie[p]\{fc}«je[wj] 

Assume that Sk / 0. Then the two rank one tensors Sk Oi^GK] y^k+ik^^k <^«fce[ajfc] 



n n " 

ie[p\\{k} 
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Zai^+if, G M(™'fe)'^"'° satisfy the assumptions of Lemma 
mu tation Uk : [wfc] [^^fc] such that ifc<^«feeK]^«fc+^fe 
of ( |7.10[ ) we deduce the equality '^it,e[ujk]'^ak+ik = =t< 
exists a; : [d] — ^ [d] which leaves invariant each set [oj+i] for j G [p — 1] such that 
<Sije[d]'^j = i Xjgj^] yo-(j)- As fXijgfdlZj and ^je^rfjyj are best rank one approximation 

□ 



Hence there exists a per- 

Sk'^ikeMy»k+-kM- In view 
eMya,+aM- Hence there 



to T we deduce that 



^je[d]'2'j 



Recall a recent result of the first author which claims that each T G S'^ (M) has a 
best rank one approximation which is a;-symmetric O Theorem 1]. For symmetric 
tensors this theorem is ]3\ Theorem 4.1]. Use Lemm a |15| Theorem [sj |5j Theorem 
1] and arguments analogous to the proof of Theorem |13|to deduce. 
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Theorem 17 Each T G S'*^(R) has a best rank one approximation which is 
u} -symmetric. Furthermore, there exists a variety W C S'*^(M) such that for each 
T G S^(M) \ W the best rank one approximation is unique and u) -symmetric. 

We now give an independent proof of this theorem which does not use the results 
in [HIE]. 

Proof of Theorem |17[ We first show the result in [3l E] that for each sym- 
metric tensor T G S'^(M") there exists a best rank one approximation which is also 
symmetric, i.e. of the form ±0'^y. We prove that by induction on d. For d = 2, i.e. 
T is a symmetric matrix, this is a well known result which can be deduced from the 
singular value decomposition of real symmetric tensors, i.e. |8l|5]. Assume that this 
claim holds for any d = > 2 and suppose that d = N + 1. Let T G S^+i(M") sat- 



isfy the assumption of Lemma 15 That is, there exist A^-|- 1 vectors yi, . . . , yat+i of 
length one and a G M such that all best rank one approximations of T are of the form 
a(g)j-g[jv+i]yo-{j)) for each permutation cj : + (a = Tx (g)jgjjv+i]yo-{j)-) 

We claim that S := a ^jg^jv+i] yj is symmetric. Assume to the contrary that S is 
not symmetric. So a 7^ 0. Consider the symmetric tensor 71 := Tx yat+i G S^(M"). 
By induction hypothesis there exists a unit vector x G M" such that b (8>^ x is a 
best rank one approximation of S. Hence 6((X'^x) Yn+i is a best rank one ap- 
proximation of T. Therefore we can assume that = x for i G [N]. Since S was 
not symmetric tensor Yn ^^x. Let 5' := T x x. Repeat the arguments for S' to 
deduce that there exists a unit vector x' G M" such that a best rank approximation 
of T of the form b' 0^ x' (8) x, where x' 7^ itx. This contradicts Lemma 15 Hence 
T has a unique best rank one approximation S which is symmetric. 



As the set of symmetric tensors satisfying the assumptions of Lemma 15 is dense 
in S''^'''^(M") it follows that each T G S^'^"'^(M") has a best rank one approximation 
which is symmetric. 

Assume now the general case that T G S'^(M), where co = (coi, ... ,ujp) and 
p > 1. Suppose first that T satisfies the assumptions of Lemma [Tsj As in the 
symmetric case we deduce that a best rank approximation of T is w-symmetric and 
unique. Since the set of such T is dense in T G S'^(M), the arguments of the proof 



of Theorem 13 yield that there exists a variety W C S'*^(M) such that for each 
T G S'*^(M) \ W the best rank one approximation is unique and a;-symmetric. Let 
T G W. Since T is a limit of tensors in S'^(M) \ it follows that a best rank one 
approximation of T can be chosen to be w-symmetric. □ 



8 Best rank-(ri, . . . ,rd) approximation 

Assume that F is a field and m = (mi, . . . , ma) G N'^. Let M = Hjed mj, Mi = ^ 
and mj = (mi, . . . , mj_i, mj+i, . . . , m^) G N'^^-'^ for i G [d]. Assume that T = 
[tn,...,id] G Denote by G F™'^^' the unfolded matrix of the tensor T in the 
mode i. That is, let Tj^k £ I^™* be the following d — 1 mode tensor. Its entries are 
for ip G [mp],p G [d] \ {k}. So j G [m^]. Then the row j of Tj 
is a tensor Tj^i viewed as a vector in F™' . Then rankj?" is the rank of the matrix 
Tj. Ti can be seen as the matrix of the contraction map ®je[d]\{i}{^^)^'' ~^ "^^^ foi' 
« G [d\. Clearly, 

rankjT < min(mi,Mi) i G [d]. (8-1) 
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Carlini and Kleppe characterized the possible occurring as in the following 
Theorem. 



Theorem 18 ([T], Theorem 7) Suppose that ri E [mj] fori E [d\. Then there 
exists T G such that rankj?" = r^ for i G [d] if and only if 

rf < Yl fj for each i e [d]. (8.2) 

We show a related argument working on any infinite field. For each i let fi be one 
minor of Tj of order min(mi, Mj) . Let / = Iljetd] /«' which is a nonzero polynomial 
in the entries of T = Let V{m) C F'" be the zero set of /. 

Theorem 19 Let m G N'^ and assume that V{ui) C F™ is defined as above. 
Then for each T G F"^ \ V{vn) the following equality holds. 

rankjT = min(mj, Mj) for i G [d]. (8-3) 



In particular for F being a infinite field, a general tensor T G F™ satisfies (8.3). 



Proof. Suppose first that m-j < Mj. We claim that the rrii tensors 71, i, . . . , Tmi,i 
are linearly independent. Suppose not. Then any rrij x m-j minor of Tj is zero. This 
contradicts the assumption that the T G F"" \ y(m). Hence rankjT = rui. Suppose 
that rrii > Mi. Let Tk^^i, . . • ,Tknj_,i be the Mi tensors which contribute to the minor 
fi. Since fi{T) / we deduce that Tki,i, . ■ ■ , Tkj^.j.,i are linearly independent. Hence 
rankjTi = Mj for each i G [d]. Since / is a nonzero polynomial, for an infinite field 



F V^(m) is a proper closed subset of F"^ in the Zariski topology. Hence (8.3) holds 



for a general tensor. □ 



On infinite fields, Theorem 18 can be proved as a consequence of Theorem 19 



Indeed, let r = (ri,...,r^) G N*^ and assume that (8.2) holds. Choose a general 
T = G F^ So ranker = ri,i e [d]. Extend V to T = G F'" by 

adding zero entries. I.e. = for ji G [ri],i G [d], and all other entries 

of T are zero. Then rankj?" = ri,i € [d]. 

In what follows we assume that F = M. Observe that the set of tensors having 
rank-(ri, . . . , r^) contains in the closure exactly all tensors of rank-(ai, . . . , a^) with 
< Tj. This closure is an algebraic variety, defined as the zero set of all the minors 
of order + 1 of Tj for i G [d]. We denote it by C^. Note that having rank (1, . . . , 1) 
is equivalent to have rank 1. 

Clearly Cr is a closed set in M"^. The best r-rank approximation of T is the 



closest tensor in Cr to T in L2 norm [4J. Corollary 11 yields. 



Theorem 20 Let m = (mi, . . . ,m(i),r = (ri, . . . ,rd) where ri G [mi] for i G 



[d] and they satisfy (8.2). Then almost all T G have a unique best r-rank 
approximation. 

Let u = (wi, . . . ^ujp) be a partition of d, m' = (m'^, . . . ,mp) and assume that 
m(cj) is defined by ( |4.4[ ). Assume that r' = (r^, . . . jT^), where r[ G [m^] for i G [p\. 
Let y{u) = (r'i,...,r'i,...,rp,...,rp. 
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Let C'j., = Cr(a;) n S'*^ . Clearly, C^, is a closed set, consisting of cj-symmetric 
tensors in M'"^'^) having rank r(cj) . 

Let T G S'*^. Then a best u symmetric r(cj)-rank approximation of T is the 



closest tensor in C , to T. Corollary 11 yields. 



Theorem 21 Let u = (wi, . . . , Wp) be a partition of d. Assume that m' = 
{m[, . . . , m'p),r' = (r'^, . . . , r^), r- G [m-i], ^ G [p] and that m(cj) satisfies (8.2). Then 



almost all T G S'*^ have a unique best ixj- symmetric r{uj)-rank approximation. 

We close our paper with the following problem. Let T G S'^. Does T have a 
best r((x;)-rank approximation which is w-symmetric? If the answer is yes, is a best 
r(a;)-rank approximation unique for almost all T G S'*^? In the previous section we 
showed that for r((x>) = (1, . . . , 1) the answers to these problems are yes. 
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